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ABSTRACT* t * . 

Twp films, "Oxygen Breathing Apparatus Type &-3 w and 
•^Damage Control Petty* Of f icer ,)* wars developed using an educational 
systems apprbach and based on 15 behaviot'al objectives, The / 
effectiveness of each film was .tested using three samples! one which 
was pretested, viewed the film, ai^d was posttestsdt* another whfch was, 
pretested and posttes^ed but did not view the film; and another "which 
was tested without viewing the film. Results showed that viewing the- 
film significantly <dmproved„ post test scores, especially for viewers 
of high intelligence, but*. the .films failed to achieve the 80 percent 
proficiency, whi*ch had" fysen desired* Though modifications will be 
iiec^ssary to -make the films meet minimal achievement standards, both 
films >showed themselves to be significant -aids in 'the instructional 
process. The report discusses -in detail the .statistical significance 
of tie test results. (EHH) % / ' - * . 
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APPElStolX A Specific Behavioral Objectives 
(SBOs) and Test Items for the 
OBA and DCPO Films. 



1.0 SUMMARY-, The -Oxygen Breathing Apparatus Type A-3 film 
(OBA.film) and Damage Control Petty Officer filra^DCPO film) 
were evaluated. for training effectiveness. Three different 
recruit groups (Exp 1, Exp 2, and Con groups} were used^te 
evaluate each filnu The Exp 1 groups were pre^teoted for 
entrance level knowledge, viewed the film, and received a a 
post-test tQ determine whether the knowledge level had 

- ' . 5 ■ 

» " .4 •■ •' ' 

improved. The Exp 2 groups were not pre-tested, but did 
view the film and received .the post- test, The Con groups, 
were pre- and post-tested but did not view the film. In 
addition, enlisted personnel with over two years of Navy 
experience who were attending* a* damage control f amiliar- 

ization course at a^DC School were tested. These more 

i ' » 

experxeneed personnel were divided' into two groups ~ DC 1 

'(••'•' 

School Exp group and DC 'School Con group. The DC School Exp 

* * * 

group was, pre- tested, viewed jfche PCPO film, and then post- 
tested. The DC School Con group Was pre- and post-tested, 
but did not view O tho DCPO film. fThe results ahowed that (a) 
recruits who viewed the films had significantly higher post- 
test ^scores than recruits who did^not view the films, (b) • 
those recruits who were pre-tested had. post-test scores 
similar to those recruits who were not pre-tested even 
though the pre- and' post- tests were duplicate*, (c) 
personnel with over two years of Navy experience had signif- 
icantly higher scored on the DCPO pre- test than the recruit 

0 , . ' " 



groups 9 (dy the post-test performance of these experienced 
personnel improved as a r result of viewing the iDCPO film , (q) 
and pre- and post- test performance by recruits for the OBA * 
film was significantly better than pre^ and post- test J 
performance by recruits for DCPO film even though both 
.recruit groups were similar in age, years of education, * ^ ^ 
reading skill level , and intelligence* The results also 
show that those .recruits who were more intelligent (higher ; 
OCT scores) , and had better reading skills; had higher post- 
test scores on both films than recruits who were less Intel- 
ligent and had poor reading skills. Intelligence, however , 
appeared^td be more important Jbhan reading skills to OBA 
post- test performance,. Strong- agreement was also found 
among education specialists who matched the pre- and post- 
test items with the "specific Behavioral Objectives (0BQg) 
thaf'were listed for each film. "Further analysis showed 
that only eight of the IS SBOs listed for the OBA filfa were 
trained tip the 80% i criterion level, while the DCPO film did 
even snore poorly, training only three of the 15 SBOs to this 
level. Despite these poor achievement levels, the conclusion 
is made that training can be more effectively managed with 
these film programs than with any other films that are 
Currently available because the use of SBOs permits accurate 
correction of inadequate scenarios > while pre-° and post- 
testing allows, for more valid measurement of achievement 



arid more effective training management..' The recommendation 
■is made that the System Approach.to 'Training (SAT) princi- 
ples that were "used in developing these film programs be 

used in managing future film procurements. 

• , * ■ V 
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3.0 PURPOSE . This report presents the results of an 



6 effectiveness evaluation of two films which were developed 
using the concepts and methodologies of ' the Systems Approa 
to Training (SAT) as presented by Havens et -al. In 



Havens, C.B., Prophet, J.E,, Thrash, N.E., and MeKibben, 
J.R. Introduction to the Systems Approach to Naval Air . 
Basic Training. CNABT P-802 PAT , Chief of Naval Air Basic 
Training, Pensacola, Florida, 1^68. 
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'addition, data on factors which may have been involved in 
the training effectiveness of these films., including reading 
skills, general intelligence, education level, "and Navy ■ 
experience , are presented. 

o » oft 

• * ■ I * 

- * - * 4* » 

4.0 BACKGROUND. This task .originated with CNETS fetter 
H44/HHK:dpg of 26 July, 1972 which requested the Naval „ 
Training Equipment Center to manage the procurement of two 
training films/ The first of these films' would .train for ' 
the operation; and /use of the Oxygen. Breathing Apparatus Type 
A-3 {referred to /as' the OB A 1 film) , and the second film would 
'train for the duties of the departmental and division 
Daraange Contra^ Petty Officer (referred to as the DCPO 
--film). This request was unique among training film: pro- 
curements in that this was the first procurement to be raade 
within 'the Naval! Training Command, and. it was the first , 
procurement to specify the development of a Film Utilization 
Guide for each film."- The purpose ^fAhis guide was to 
provide the instructor with information to, better manage the 
film presentation \f or training purposes. . # » 

Approval of tw film request was provided in CNO letter 
OP-991Q/js Ser 835 tP991 of 14 Dec 1972. Procurement of 
these' films was to be made on a pilot basis in order to 
determine whether the improved training methodologies and 
techniques; available^ within the Naval Training Command would. 



result in films that we^a f, pedag©gically more e^fQ<rtivQ" C * 
than f ^mg procured from other sources. . , >. :.? v 

Although these films had been evaluated for .technical , s 
accuracy by the Chief of Naval Teahnioal Training/ the 
training (or pedagogical) effectiveness of. the films rq« \ 

mined to>Jbe demonstrated; A request for an effectiveness * 

« « « , 

» * 

evaluation was therefore sent to the Naval Education and 
Training Support Center &a6ifia (NAVEDTRASUPPCENPAC ) in ." 
CNETS letter Code' N~2131:mae of 1 Nov 1974. The results of 

' <f ■ 1 . % 

this evaluation, as described in the following paragraphs, 
were submitted ,in NAVEDTRASUPPCENPAC Xtr Nl:PW: jmh 1591-3 
ser 601 of 25 Apr 1975. 

• / • • ■ • - : ; •• } ■ 

5.0 METHODS . The- following sub' jects, tests, .procedures, and 
statistical analyses wqro used in perf owning the evaluation* 

5.1 Subjects . Eight different groups of subjects were, 
used in this" evaluation. All groups consisted of male & 
enlisted personnel who were on active duty in the U,S Navy* 
Three groups participated in the evaluation of the OBA film, 
and five groups we«q used to evaluate the DCF0« films. 

Table 1 presents a summary of the groups acid conditions 
used in this evaluation. As this table shows, the three 
groups involved in the OBA evaluation consisted of Navy 
recruits who were in the third' week of training at the' 5 
Recruit Training Command, San Diego, California. Two 

• •'' . 12 • 

' . . '. .5 



experimental groups and a single control group were tested, ° 
s The ;two experimental groups received the film-, while the 

•• ' * V • ; ■ i ■ - • , , ■ 

control group did not. The first experimental group (OBA 
Exp 1) was administered a pre-test before viewing the film. 
Immediately after the film, they were administered a post- 

■ . ''' Q • - 

test. The second experimental <;group (OBA Exp 2) 'did not 
receive a pre-test hut was administered the post-test after 
viewing the film. *The third group was a control. group (OBA 
Con) which was Administered both the pre- and post-tests,, 
but did not 'view the film between tests.' 
V Further examination of Table 1 shows that three of the 
' five 'groups used in- the DCPO^film evaluation consisted of 
Navy recruits who were tested under the same conditions as • 
the three OBA recruit groups. These three groups wi^l.be , 
ref erred to as the DCPO Exp 1, DCPO Exp -2, and DCPQ Con 
groups. The two remaining groups consisted of students who 
were" attending a. 1-day familiarization course at the Damage. 
Control (DC) School, Service School Command, San Diego. 

. X *' ■■■ '•..''.."'.'«••.:.."*. 

These were enlisted male personnel who were permanently 
attached to local commands and were attending this course' to 
review basic damage control procedures . They represented 
several different ratings (mostly BTs, BHs, ENs, HTs, and 
RMs) and pay grades. The data from these last two groups. ? 
would provide some information on the effectiveness of this 
film among more experienced ^Navy enlisted personnel. The 



first of these two groups took the pre- test , viewed the 
film, and was then .administered the post-test,, while the 
second group did not view the film but took both tests. The 
first group will be referred to as the DC School Exp group, 
and the second, group will be identified as the DC School Con 
group. <> 1 * • ^ ; 

Table 2 lists the average age, years of education f / 
General Classification Test (GCT) scores f and reading skill 
levels (Gates-MacGinitie, vocabulary * and comprehension^ seer 

* • . ■ . ' ■ .-«•'- 

tions) of the three recruit groups who participated in the 
OBA and DCPO evaluations; 7 and the average age, education 

level, years in the Navy, pay grade, and months in pay grade 

- , . . & ■■■ . 

of the two DC School .groups who were subjects in the DCPO 
evaluation. d 

5*2 Film and Test Development . As mentioned above in 
paragraph 4.0, these films were unique because -the training 

methods and techniques of the Systems Approach to Training 

•'• . - i . . •••• t* -. 

were used to develop the film programs . These programs 

established a set of 15 Specific Behavioral Objectives 

(SBOsL-to be trained by each film. The film scenarios were 

subsequently -designed around these SBOS. The SBOs are ; 

presented in Appendix A in the order listed in the Film - ; 

' •..■/. ' " . 

Utilization Guides. ' 

•' ' . ' ■ . 1 . " 

In keeping with. SAT principles, pre- and post-testing 
questionnaires and test administration procedures were also 



designed into the * film programs. The pre-test, /which was 
administered before viewing the films, measured ijhe entrance 
leveM knowledge which the subjects had of each SBO. Pol- 
lowing training, a post*-test was administered in order per 
measure SBO achievement which resulted 7 ^ from viewing the 

'% / » . 

films. A list of the test items for each film, as published 
in the Film utilization Guides ,-As presented in Appendix A. 
The Film\utilization Guides were also designed to improve , . 
management of training. The guides identified the appropriate 
audience, yListed the terminal objectives to^ meption in 
introducing the^f ilm, described the pre/post test adminis- 
tration procedures, and explained the usefulness of pre- and 
post*- tasting as Measures of training effectiveness. The 
following materiel numbers (MN) were assigned to the film 
programs by the Naval Photographic Center: MN 11369 (OBA 
film) and MN ,11370 (DCPO film); ... ./ ' * \ 

5.3 Evaluat ion Procedures . The recruit subjects were 

—i — v . • • 

randomly divided into three groups (Exp 1, Exp 2, and Con) 

■ ■ .. .. -'\ . K ' 

for each of the two filmsi and taken to separate classrooms. . 

The pre-tests were administered to the Exp 1 and Con groups, „ 

followed by introduction and presentation of the films to*' 

the Exp 1 and Exp 2 groups. The Con groups were allowed 

free discussion during this period. The post-test was then 

administered to all groups. These procedures were also used 

to test the txto DC school groups* ' These" £?C subjects were 



tested and shown' the ffli^s grior* to the start of the familiar- 

\ . ;'■ • / / 

ization course* - 

■'• ■ ° < • • 

\ 5 • 4 Matching SBOs and Test Items , in order to deter- 

mine whether the SK^were accurately represented by the . 

\ 

te£t items, a group of, five education, specialists (ps 1710 
series) , at or above the GS-12 level , were aske£ to match 
each test item to the^SBO which they judged to be the most 
appropriate • Half* of this group matched the SBOs and»«f£est, ; 
items for the OBA film first^ followed by matching tojc the 



DCPO film, while the other half matched for the DCPO film 

\ 

first \and then the OBA film. 

S . 5\ Statistical Analyses A one-way analysis of 
variance \ANOVA) for unequal n f s was used to determine > 
whether significant differences existed across groups for 
pre- test and post- test scores, or f<^r such factors as \ age, 
education, reading skill levels, and so forth. Comparisons 
between groups were made using t-tests for uncorrelated 
samples. The relationship of several aptitude and demc?- % 
graphic factors to post- test scores was determined using 
Pearson product-moment correlations. A* multiple regression 
analysis was performed in order to determine which of these 
correlations accounted for unique post- test variance. The 

■ » Q " 

significance level was established at pA05 (two-tailed) . 



6.0 RESULTS . The following differences and correlations 

' — * <3> : 
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were found among groups, and conditions. 

6.jjL Agreement . Between SBOs and Test Items . Hatching of 

SBOs afid test items by the education specialists was in 

H ■ / ' ■ . ' i ■* '■■■■*•'■ . - .; *" 

agreetfjent in . 93.33% of the) cases for the OBA film, while for 

the D|jPO film the level of agreement was 98.67%. Each film 

had a| 'total of 75' judgments to bje made by the five education 

specialists. Five of these judgments were in disagreement 

w • 1 / 

for tate OBA film, while onjLy jk single disagreement occurred 
for title DCPO '£ ilm. Table 3/ shows which SBOs were judged by 
^ most pi the education specialists to roatcil each test item .. . 
f or t|e two films. As this table shows, tine listing of SBOs 

- ■ \\\ '.' . ' / , •■ ' „ 

in the Film utilization Guide for the OBA film does not 
correspond to the listing of test items, indicating that- the 
SBOs and test items were not cross-referenced in the OBA - ( 
guide. 

6.2 Differences ^Among Aptitude and .Demographic Factors . 
The ANOVAs did not show any significance differences among 
the three OBA recruit groups (Exp' 1, Exp 2, and Con groups) 
in GCT scores, reading skill levels, age or years of edu- 
cation. Years of education did not differ among the five 
dcpo groups (three recruit groups and two DC School groups), 
although age was found to differ significantly among the 
five groups . <F«1€ . 963 ,* ' di*A/ph ; p<. 01. A series of t-tests 
'showed that the three DG1?© recruit groups did not' differ' 
Significantly in age, nor did the two DC School group*. The 

. . • - . 17 . 
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mean age of the combined' DCPO recruit groups, however,, did 

* * 

differ significantly from the mean age of the combined DC 
School groups (see Table 4) # with the DC School group being 
older. Neither of the two DC School groups differed signifi 
eantly from each other in years « in Navy, pay grade, or 0 
months in pay grade. Soma differences in QCfa scores and 
reading skill levels were found, however, among the three 
pppO recruit groups. Although the DCPO Exp 1 and Con Groups 
had similar GCT scores, Table 4 shows that these two groups 
combined had significantly lower GCT scores than the DCPO 
Exp 2 group. In addition, Table 4 shows that the reading 
skill level, of the DCPO Con group was significantly lower 
than the reading skill level of the DCPO Exp 1 group, while 
the reading skill level of the DCPO Exp -ITgroup was signifi- 
cantly lower than that of ythe DCPO Exp 2 group. 

6.3 Differences in Pre- and Pdst-.Test Scores?/ The 
ANOVA for the pre-test scores of the five DCPO groups re- v 
Suited in a significant P {F « 137.707* df*3/202j p<.01) ♦ 

■ ' ' ' - 

Subsequent t^tests. did not show any significant differences 
among the two recruit groups which received pre-tests (DCPO 
Exp 1 and Con groups), or between the two DC School groups. 
As shown in Table 4, the combined recruits groups did* 
however, differ significantly for» the combined DC School 
groups, with the DC School groups having significantly 
higher pre-test scores. 

18 



The ANOVA found that post-test scores also, varied 
significantly among the five DCPO groups (P=8i;8l9; df«4/237,; . 
p<.01). T-tests presented in Table 4 show that the DCPO Exp 

1 group had significantly lower post-test scores than the 

* * • 

DCPO Exp 2 group and significantly. higher post-test scores 
than the DCPO Con group/ Table 4 shows further that* the 
post-test scores of the DC School Exp group were significantly 
higher than the post-test scores than the DC School Con 
group. > In addition ft the t-tests in Table 4 indicate that 
the DC School Exp group had significantly higher post-test 
scores than the DCPO Exp 1 recruit group/ 

A t-test showed£hat pre^test performance of the OBA Exp 
1 and Con recruit groups was similar, whil<^ an ANOVA showed 
that post-test performance did vary significantly among the 
three OBA groups (F=238.9; df =2/198; p<.01). The OBA Exp 1 
and Exp 2 groups did not differ in post- test performance, 
while the post-test scores of these two groups combined were 
significantly higher than the post-test scores of the OBA 
Con group (see Table 4). Further data presented in Table 4 
show that both' the pre- and post- test performance of the OBA 
Exp 1 recruit group differed significantly from the pre- and 
•post-test performance of the DCPO Exp 1 recruit gr^up. The 
pre- and post-test scores of the OBA Exp 1 ^group were 
significantly higher than the scores of the DCPO -Exp 1 
group. The two groups did not differ significantly in age, 

- y 19 \ • 
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years of education, GCT scores, or reading skill levels* 

Table 5 presents summary statistics for the' pre- and post- 

test scores of the three DBA and* five DCPO groups, 
■ • * * - ' > * ' • /" 

6*4 Correlations Between Aptitude and Demographic/ 

Factors and Post-Test Scores . * Regression analysis of' GCT 

scores, reading skill levels, age, and years of education 

wxth post-test ° scores for the combined OBA Exp l/and Exp 2 

* " ■" . * • / . ... 
recruit groups^ showed that post-test performance was signif- 
icantly associated with GCT scores (r ~ .346? df=136? p<.01) 

and reading skill 'level' ( r « .26; df=136? p</oi) . The 

• ■ . ' * " » • / . 
multiple 1 R was .351, with GCT scores accounting for most of 

' . " : ° . ° > ■ * . /' - • 

the post- test performance variance, GCT scores had a beta 
weight of .300 .and reading skill level had a, beta weight of 
.074,. * These results were; replicated in . the combined DCPO 
Exp 1 and- Exp 2 recruit groups. For these ji>CP0 groups, GCT 
scores correlated .401 with post-test performance (df-129; p< 
.01)', while /reading skill level correlated . 45 with post- 
test scores, (df«129; p<.01). The multiple R resulting frojn 
-these two factors was .466, with the beta Weight for GCT 
being .174, while thebejta weight for reading skill level 
wad .324. 

A regression analysis was also performed on the jre- 
° lationship between the ppst-test performance of the DC 
School Exp group and several factors, including age, years of 
education, years in Navy, pay grade, and months in pay 



grade. Only months in pay grade was significantly associated 

/ ' " w / ; . ■ '• . - 

with post-test scores ( r*,423j df»22; jp£.05) .. e 

6.15 Achievement levels for SBOs . An analysis, was also 
made of the percentage of combined Exp 1 and Exp 2 recruit 
groups who met each of the 15 SBOs developed for, the two 
films. This analysis showed tha€ achievement varied sub- 
stantially between the two sets of SBOs. If a 95% achieve- 
ment criterion* were established for an SBO (that is, 95% of 
the trainees had to get the appropriate test item correct), , 
then only two SBOs in the OBA film - (Hop . 2 and 14) would 
have been satisfactorily trained, while none of the SBOs for 
the DCPO film met this criterion. Adoption of a 90% criterion 
would add only a single SBO to those met by the OBA film 
(No. 15), while none of the SBOs for the DCPO film would 
have reached this level. An additional five SBOs fNos. 1, 
7, 9, 11, and 13) would have been met by the OBA^fllm if an 
80% criterion were established, while only three' SBOs (Nos. 
6, '?, and 10) would have been trained to this level by the 
dcpo film. -* , ' .. 

7.0 discussion . These results indicate that significant 
overall improvements occurred in knowledge of the A*3 Oxygen 
Breathing Apparatus and duties of Damage Control Petty 
Officer as a result of viewing the OBA and DCPO films. Im- 
provements in DCPO post-test performance were found for both 
experienced enlisted personnel who had an average of over . 

•' ' ; • 21 . ■ .. : 

o Z4 
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two years* of Naval service and inexperienced reqruits • ^ 
Post-test performance also appears to be related to intel- 

ligence and reading skills.- in addition , strong agreement 

• ■ * , 

was found between the SBOs and test items as judged by r 
professional education specialists, although the sequence of 
SBOs and test items listed for the QBA film in the Film 
Utilization Guide were dissimilar. This inconsistency could 

.... V 

prove confusing to instructors who are trying to. identify 

a, 

specific training deficiencies through post- test scores. 
f The recruits appeared to have a higher entrance knowl- 
edge level for the OBA information than for the DCPO infor- 
mation.^ In addition, the knowledge of those recruits who 
were trained oh the OBA film improved more than the knowledge 
of those who were/ trained on the DGP0 film. None of these 
pre- or post- test differences between the OBA and DCPO 
■ recruit groups were related to differences in intelligence 
(GCT) , reading skill, age, or years of education. It was 
also found that experienced ^persor^nel who viewed the DCPO 
film entered at significantly higher knowledge levels *than 
the recruit groups r and had significantly higher post-test 
scores as well. These results indicate that over two years 
of Navy experience have been effective in training many of - 
the SBOs, and that the film was effective in significantly 
improving knowledge of damage control procedures among the 

experienced trainees . * . 

» 

22 

15 * 



«r The results also demonstrated that exposure to the pre- ■ 
test did not modify performance on the post-test. For those 
recruits who viewed the DCPO film, the reverse Was true-- the 
post-test performance of those who were pre-tested was worse 
than the post- test performance of those who did not receive 
the prertest. The better pogt-test performance of those who 
were not pre-tested is probably related to the higher GCT 
scores and reading skill levels of thin group. The absence 
of better post-test performance among the pre-tested groups 
is an important finding because the pre-and past^tests were 
duplicates, and were administered about 30 minutes apart. 
The. probability appeared high that the pre-tested subjects 
would have had a learning advantage because they may have 
been more familiar with the content of the SBOs. The 
failure to show a pre-testing advantage makes i£ unnecessary 
to design separate post-tests in order td measure the 
training effectiveness <of films. 

* V These results also show that post-test performance among 
recruits on both films was related to 6CT scores and reading 
skill levels. Those who had higher verbal intelligence and 
better reading skills had higher post-test ^scores and ap- 
parently learned more from the films. Reading skills appeared 
to be more important to DCPO post- test performance -than to 
performance on the OBA post-test. For the more experienced' 
Navy personnel (DC School group) , months in pay grade was 

' .„..•;../• ' 23 v 
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n ■ : 

associated with post-test performance, -Being in, pay grade 
longer is indicative of more experience in grade, and this 
experience nay have provided this group with a learning 
advantage while viewing the OGPO film, » 
Although post^test comparisons between experimental and" 
control groups showed that ; the films, significantly improved 
subject matter Knowledge , the lev^i q£ i&provement did not 
meet the minimal aehievemant criterion <8w§) usually adopted 
for training-. This was especially fcsnve qf the OCPO film. 
The present evaluation does demonstrate, however, that 
training can be managed more effectively with these films 
. than with any other films that are currently in the training 
inventory. Effective* training management was made possible 
by developing a aet pf° SBOs, and measuring SBO achievement 
through pre- and post-testing. Both individual learning and 
system operation can be more precisely and effectively 
managed on the basis of these post-test data than is pos- 
sible with any other available program. This precision 
could reduce unnecessary retraining of individuals as well 
as unnecessary modifications in the training system , thereby 
resulting in improved training at less cost. On the < basis \ 
of the present Evaluation, for example, the film scenarios 

related to those SBOs which had low overall achievement 

| * . ■ ■ ' *' ' ' 

levels can now be identified, corrected f and re-evaluated. 

• f ' -¥ . - o * ; 

Although secondary to the above evaluation, the fol- 



lowing observations- may hel&\ improve the training ef- 
fectiveness ©fxfchese films . 'ffl review of the Film utilization 
Guides indicates that some revision may be necessary if 

* i\\ '- * o 

» ft- W . 

instructor^ are to understand and follow the training ; 
• 0 principles involved. Instructors should be provided with a 
more thorough explanation of the pre- and post-tests, 

• • * ; 

especially the association between test items and SBOs, and 
the ways in which individual training deficiencies can be 
^identified from post-teg* scores. As mentioxpd above in 
f . paragraph 6*1, the list of the SBOs and test items should be 
accurately cross-referenced. In addition , the pre-test h 
should be used to identify those who have met the SBO 
achievement levels |ahd will not have to participate in film 
training. 

o 

8.0 CONCLUSIONS . The following conclusions are made from 

the above results: ''']/"' 
8.1 Recruits who viewed the films had" significantly > 
1 higher p©3t-test scores than recruits who did ( not view the 

J films';.' 

8.2. personnel with over two years of Naval service had 

significantly higher scores on the damage^control pre-test 

, than the recruit groups . 
. * * " ." ■ •. 

*. 8*3 The post-test scores of these experienced personnel 

improved significantly as a result of viewing the damage 



FRir 



18 



control film* • * , , 

8.4 The pre- and post^test performance of recruits who 
viewed the oxygen, breathing apparatus film were signifi- 
cantly higher than the pre-* and post-test scores of the 
recruits who viewed the damage control film. 

8.5 post-test performance by recruits was significantly 

related to intelligence and reading skill level. 

■ . " ■ i . •'" ' .. V- ■• ■ ' •• 

.8.6 Pre-testihg did not/appear to modify post-test r 

performance. ■•- 

8.7 Education specialists showed high agreement in 

\ t - - •"" • . 

matching test items and Specific Behavioral Objectives 

8.8 jThe post-test SBO achie^ment levels attained by 

i below normally accepted,. 



recruits who viewed the films we 
standards. V 

• ■ v ■ ' ■■ ^ -x - - , . 

9.0 RECOMMENDATIONS . The following recommendations result 

_ , — v , ; ;. v .; . 

from the above conclusions. iff 

9.1 The films would be. useful in training both recruits 
and experienced Navy personnel. ^ \ 

9.2 some, modifications in the film scenarios will be 
necessary if the films are to meet minimum achievement " 

; standards. . ... \ V. 

I' \ 9,3 These modifications will be easier than with other 
current films, because the use of SBOs permits accurate v 



■ ■ : \ . • . • ■ . . , • 

identification of inadequate scenarios, 

9/4 The Film Utilization Guides, especially for the 

"oxygen breathing apparatus film, should cross-reference .SBOs 
and test items. In addition, a more thorough explanation of 
the purpose of SBOs and pre/post-testing should be provided 
so that instructors can^h^jp? effectively manage the training 

* situation. 

9/5 Duplicate pre- and post-tests can be used without 
modifying the usefulness, of these measures. 

9.6 Less intelligent viewers or viewers with poor 
readirig skills may have to be provided with additional or 
supplemental instruction 0 if minimal achievement - levels are 
to be attained* * ■ * : * 
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TABLE 1 r 

Summary of Evaluation D^ign, 



Group j 
Name 



N 



Group . Viewed 
Composition % film? 



Tests 
Taken 



OBA 


OBA Exp 1 « 


> 94 


Recruits 


Yes 

0 


Pre, v Post 








^ecruxts 


Yes 


Post only 




OBA Con 


', 63 


Recruits' 1 f" 


No 


Pre, Pos£ 


DCpO 


DCPO Exp 1 

s 

DCPO Exp 2 




Recruits 


i Yes 


pre, Post 




36 


Recruits 


Yes 


Post only 




DCPO Con \ 


73 


k . Recruits 


No 


Pre, Post 


DCPQ 


DC scnoox Exp 1 


23 


Experienced* 


Yes 


pre, post 




DC School Con 


17 

- • s 


Experienced 


No 


v Pre, Post 



Iveraged over two years of Naval service. 
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TABLE 2 



Demographic- and Aptitude Factors for 
Different Subject Groups. 

Recruit .Groups 



Subject 
Group 



OB A Exp*! 
OBA Exp 2 
DBA Con 
DCPO Exp 1 
DCPO Exp 2 
DCPO Dan 




years 
Education 



. GCT 
Scores 



Reading 
Skill level 



sd 



sd 



18.94 
20.09 
18.59 
18.88 
19..33 
19.30 



1.90 
2.'44 
1.41 
1.88 
2.22 
2.21 



11.55 

11.90 

11.37 

11.42 

11.61. 

11.71 



1.11 

1.26 

1.23 

1.45 s 

0.89 

1.43 



. 52.66 
55.20 
51.70 
51.65 

\56.13 
49.40 



9.72 
7.55 
9.09 
8.25 
7,44 
8,48 



10.06 
10.31 
9.87 
9.64 
10.61 
8.95 



2.15 

1.91, 

1.89 

2.10, 

1*39 

2.33 









* * 












DC School Groups < 






• X 


Subject 
Group 


Age 


Years 
Education 

n 1 


Years in 
Navy 


Pay 
Grade 


Months 1n 
Pay Grade 


m 




•1 sd 


. X ' W 


' X sd 


X sd 


X ^ ' 




DC School Exp 
DC School Con 


21.30 2.74 
23.65 4.65 


11.93 1.07 
12.47 1.33 


2.99 2.84 
4;80 4.73 


.3.74 0.99 
4.12 1.18 


16.10 13.2^ 
36.02 47.15 





TABLE 3 



Matching of SBOs and Test Items : 
by Education Specialists,* 

OBA Film '• DCPO Film- - % 



* 

SBO No. 


Matching 
Test Item No. 


; SBO No. . 


Matching 
Test Item No. 


1 


1 


1 


' 1 


2 


2 


2 


2 


. 3- . 


4 - 


3 


. 3 


4 •" 


5. • 


4 , 


4 


5 


. 6 ••• » 


5 


5 - 


: 6 . 


8 • 


■■■4 S 


6 


7 , 


'• •' 3. ■ . 




7 


6 


• ■ 9 " - ' 


8 k ' 




9 


.. : . IT ■ 


9 ' 


% : 


10 


12 ^ 


, . 10 . 


10 e 


11 


7 


11 


11 


12 


10 


12 


12 


13 


13' 


13 


. 13 


14 


14 


; 14 ' 


14 


16 


15 


•15 


..15 



0 



* The test item judged by the majority of the education specialists 
as being most appropriate fonfhat SBO.. For a listing of SBOs 
t arid test items, see Appendix A. * 
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TABLE 5 

Sunmary Statistics for Pre- and 
Post-Test Scores of Different Subject Groups, 



Subject 
Group 



OBA Exp 1 
OBA Exp 2 
OBA Con 
DCPO Exp 1 
DCPO Exp 2 
DCPO Con 
DC School Exp 
DC School Con 



Pro-Test 



Post-Test 





Scores 




Scores 




N 


. - X / . 


sd 


X 


sd 


94 


4.55 


1,83 


12.34 


2.73 


44 






12.59 


2.55 


63 


4.03 


1.97 


4.02 


2;11 


93 


^,,5,63 


2.00 


9.46 


2,54 


36 






10.50 


2.29 


73 


5.38 


1V73 V 


5.97 


1.86 


23 


11.96 


1.49 


14.00 


1.53 


17 


12.18 


1.34 


12.82 


1.72 



V 

% 
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Specific Behavioral Objectives (SBOs) 

and ' 

Test Items for the OBA and DCPO Films. 
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OB A Film 

Specific .Behavioral Objectives (SBOs) 



• Contgnt 

Identifying the primary use of the OB A 

Identifying the source of oxygen ' 

Recalling by what method oxygen is cooled 

Describing how oxygen is supplied 

'Naming the elements needed to produce oxygen 

Indicating how eyepieces are kept clear of fogging a 

Indicating how system is put into operation 

Identifying methodised to determine when system 
is running out of oxygen 

Name maximum range of -time that system is designed 
to be effective 

I ' • - 

Describe manner in which the system is started 
manually . 

Indicating proper method of resupplying oxygen 

Recognizing that the oxygen canister contains caustic 
chemicals 

Recalling that the oxygen supply in contact with oil 
will create a hazard . .. 

Determining proper environment fofl^Je of OBA 

Identifying what will cause system to fail a 
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Test Items 



Content 



The Oxygen Breathing Apparatus Type A-3 is 
primarily used for: 

A. Protection against heat ana fire. 

B. Protection against biological and chemical % 
agents. ' 

C. Protection against smoke and similar irritants. 

D. Protection in underwater environment. 

The Oxygen Breathing -Apparatus Type A-3 uses, the 
following source of oxygen: 

i 

A. Bottled Oxygen. ' ° 

B. Cdmpressed Air Tank. , 

C. Oxygen Generating Canister. 

D. Air Fitter System. 

The OBA Type A-3 system is put into, operation by: 

A. Opening oxygen tank valve. 

B. Inserting gas filter and starting timer. 

C. Energizing air compressor, 
p.' Pulling quick-start lanyard. 

The oxygen produced by the OBA system is cooled 
by: 

A. - A metal heat exchanger. . 

B. No special means. 

%. The flow of oxygen through a breathing bag. 
( D. A chemical process. - 1 

' O ' 

The oxygen supplied by the OBA depends on: 

A. Air pressure of lOpsi. - 

B. An oxygen line. s 

C. A charcoal filter. ? . , 
0. A chemical oxygen generator. 

The continuous supply of oxygen, in the OBA 
system prinfirily depends on the presence of: 

A. Carbon dioxide, water vapor, and chemicals, 

B. A storage tank of oxygen. 

C. A chemical filter. 
0., An ozone inhilator. 

35 



Content 

Personnel wearing the OBA in a smoke-filled 
compar**ent may safely get a resupply of oxygen 
by: 

A. Quickly exchanging tanks* 

B* Using the buddy system. 

C. Switching to the reserve supply. 

0. Going to fresh air environment for resupply. 

The eyepieces on the OBA fasepiece aye kept clear 
of fogging by: 

A. Special anti fog chemicals in' the system. 

B. Thermal glass. ' ... 

C. Air flow only. 

D. Air drying. filter. 

Where the OBA ne'ars the end of its useful 
supply of oxygen, the wearer will notice: 

A. A smell of smoke or irritants in the. face- 
pieces 

B. A red colored signal in the flow meter. 

C. Eyepiece fogging. " ? 

D. Collapse of air breathing tubes. 

One of the possible hazards relating to the 
OBA is: . 

A. The handling and disposal of caustic chemical: 

B. The overheating of charcoal filters. 

C. The rupturing of the oxygen pressure. line. 

D. Inoperative flow meter. 

The OBA is designed to supply oxygen for a period 
of approximately: 

A. 10 to 20 minutes, 

B. 45 to 60 minutes. 
•C. 90 to 120 minutes. 
D. 3 hours. 

In crde* tjo manually start the OBA system, the 
wearer must: 

* 

A. Open the oxygen valve. 

B. Use own breathy to fill the system. 

C. Connect the compressor line. 

D. Bypass the filter pack. 
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Content 

The QBA can present a hazard if the oxygen 
supply comes irt contact with: " 

* " ' 

A, Oil. ' 
b! Silicone. 

C, Charcoal inhibitors, 

D. Chemical in the standard fire extinguishing 
bottles. 

The DBA works safely under water in depths up to: 

A. 25 feet. 

B. 40 feet, * 

C. 90 feet. « 

D. None of these depths. 

The OBA is a reliable system; however the system 
will fail to function if: 

A. The canister is inserted backwards. 

B. The oxygen valve is closed. 

C. The heat exchanger overheats. . 

D. The flow meter is in the off position. 
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OCPD'PMwi 

Specific Behavioral Objectives (SBOs) 



CONTENT 

j<v.. 

Identifying the Department responsible for Damage 
Control . 

Naming the' Damage Control Officer. / 

Naming the officer that he 1s directly assigned to. 

Designating the Damage Control Officer Coordinator. 

Selecting three duties df the Damage Control Petty 
Officer. * . — 

Defining purpose of Material Condition of Readiness. 

* :v- 

Identifying terms used In Material Conditions of 
Readiness. 

Recalling condition when ship is under maximum 
watertight security. 

Defining a condition of Modified Material Readiness. 

Naming the area where listing is found. 

Identifying position of a Williams fitting during 
conditions of readiness. 

Designating when a Dog Zebra fitting Is secure^. 
Identifying purpose W 3M System. 
Defining what a MRC card is. " 

Designating which system Is responsible for preventive 
maintenance. 
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■ • Test Items ■ V 

CONTENT 

The overall Damage Control Organization is the *■ 
responsibility of: 

A. The Engineering Department. 

B. The Ordinance Department. 

C. The HUH Technicians. 

D. The Division Officers. ^ 

The Damage Control Assistant works directly for the 
Damage Control Officer who is: ^ 

A. The Commanding Officer. 

B. The Executive. Officer. 

C. The. Engineering Officer. - . 

D. The Fire Control Officer. 

Damage Control Petty Officers are assigned directly 
to the: % . 

■ XL . ' 

A. Commanding Officer. , 

B. Department Heads. 

C. Division Officer. 

D. . Engineer Officer. . * . 

A Damage Control Petty Officer works in close 
coordination with: 

A. The Damage Control Assistant. 

B. The Officer of the Deck. - - 

C. The Datage Control Monitor. 

D. The Executive. Officer. ^ 

Duties of the Damage Control .Petty Officer include: 

A. Train and instruct division personnel on damage 
control procedures. 
" B. "Inspect damage control equipment in assigned spaces. 
G. Maintain and update list of damage control fittings 

for assigned area. 
D. All of the above duties. 

MateriaVCondition of Readiness has to do with: 

& 

A. The maintenance and storage of ship supplies. 

B. Inspection of munitions and stores. 

C. The protection of the ship against fire and flooding. 

D. Combat readiness of weapon stations. 



COHTENT 

« terns X-Ray, Yoke, and Zebra are: 

A. Primarily phonetic characters for radio communication 
transmission. . '. 

B. Used to Identify the setting of different degrees of 
watertightness of the ship. 

C. Never use* except during combat or call to general 
quarters. 

D. Terms used during radar search and early warning 
exercises. '. 

Maximum security of the ship is 'provided when the 
following condition is in effect: / . - ! 

A. X-ftay. , • 

B. Zebra, . 

C. Yoke. * • * 

D. W1lco. 

Modified Material Conditions of Readiness has to do 

with;... 

*■"'■■■' . . r .." ' 

A. Allowing movement of personnel through access Ways 
of the ship without asking for special permission. 

B. Storage of dangerous munitions in standby compartments; 

C. Reduced radar watch during night time operations. 
0* Refueling. operations at sea. . 

A list of damage control fittings for an assigned area 
are found: 

a. " • 

A. In the work center lockers. 

8. On a compartment check off list. 

C. In the Executive Officer's Office. 

-D, In the general equipment ;1ndex file. *' ' 

During conditions of readiness X-Ray,' Yoke, or Zebra 
a Will iams fitting is: 

a! Kept open. 

B. Automatically closed. 

C; Closed by hand. « 

D. Closed only during condition Zebra. 

Fittings marked with a red Z inside of a black D is 
a Dog Zebra fitting which indicates: 

A. The fitting may be secured only by the Duty Officer 
during condition Zebra. 

B. The fitting must be secured during darken ship 
conditions as well as during, Zebra conditions. 

C. The fitting must be closed during daylight hours 
' during condition Zebra when at sea. & 



NO. - • CONTENT I 

1? (cont'd) VD. the fitting may .be opened during condition Zebra 

> • • for ventilation purposes. 

■ »° ■ ' • ■ .. •• -" ' * " ■» 

13 * The 3-M System used by the Navy has to do with: . 

A. Manpower. "* T ?V- 
.0. Monitoring munitions management. 
„ C. Preventative Maintenance. . r" - 

D. Modifying and malntainlngmodules. " s. 

14 «■ ... An MRC card 1s a: 

/> • ■ »• 

' * 0 , . A.xMorning Report' Card*. 
- x B. Maintenance Requirement Card. , ' 

C. Material Request Card. • 

D. Modification Requirement Code Card. ^ 

,15 V \ The step-by-step procedure for preventative 
» % '■- maintenance is provided by: 



A. The Department officers. 

B. The Division officers. 

C. The Division Chief Petty Officer, 

D. The Maintenance and Material Management System. 
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T^e Oj^gen Breathlng^Apparatus >fype A-3 film, (OBA film) and Damage Control Pettj 
Officer film XDCPO film) were evaluated for training effectiveness. Three 
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d1f I e « nt meruit groups (Exp 1, Exp Z, and Con groups) were used to evaluate 
each film. The Exp 1 groups were pretested for entrance level knowledge, 
viewed the film; and received a post- test to determine whether the knowledge 
|level had improved. The Exp -2 groups were not pre- tested, but did view the 
film and^, received the post-test. The Con groups were pre- and post-tested^^ut 
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20; / did not view the fiYm. In addition*^ enlisted personnel with over two I 
years of Navy experience who w ft ere attending a damage control familiarization J 
course at a IDC School v/ere tested. These more experienced* personnel were 
divided into two groups — DC School Exp group and DC School Con group. The 
, DC School Exp .group was pre-tested, viewed the QCPO film, and then post-tested. 
The DC School Con group was pre- and post~tested, but did not view the DCPO 
film. The results showed that (a) recruits who viewed the films had signifi- 
cantly higher post-test scores than recruits who did not view the films, (b) 
; fhose recruits who were pre-tested had post- test scores simiiar to those 
recruits who were not pre-tested even though the pre- and post-tests were '• 
duplicates, (c) personnel with over two years-of Navy experience had signifi- 
cantly higher scores on the DCPO pre-test than thg recruit groups*^} the 
post- test performance of these "experienced- personnel improved as a result of 
viewing the DCPO film, (e)l and pre- and post-test performance by recruits 
for the OBA film wag significantly better than pre- and ppst-test performance 
by recruits for^DCPO film even though both recruit groups were similar in age, 
years of education, reading skill level, and intelligence. The results a>so v 
show that those recruits who were more intelligent- (higher GCT scores), and 
had better reading skills, had higher , post-test scores on both films tham 
recruits who were less intelligent and had poor reading skills. Intelligence, 
however, appeared to be more important than reading skills to OBA post- test 
performance. Strong agreement was also found among education specialists who 
matched the pre- and post-test items with the Specific Behavioral Objectives / 
(SBOs) that were listejf for each film. Further analysis showed that only 
eight of the 15 SBOs listed for the DBA film were trained to the 80$. criterion 
level, while the DCPO film did even/more poorly, training only three of the 
15- SBOs to this level. Despite these ^poor achievement levels, the conclusion 
xis made that training can be more effectively managed with these film 4 
programs than with any otFter fiW that are currently available because the 
useTof SBOs permits accurate correction of inadequate scenarios, while pre- ' 
and post- testing allows fori more valid measurement 4 of achievement and more 
effective training management. /The recommendation is made that the System 
Approach to Training (SAT) nrtnciples that were used in developing these 
film programs be used in managing future film procurements. 
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